Numerical methods for optimal control of piecewise deterministic Markov processes

Scientific Research context: In 1980, M.H.A. Davis [1] introduced in probability theory Piecewise Deterministic Markov Processes (PDMP) as a general class of models suitable for formulating optimization problems in queuing and inventory systems, maintenance-replacement models, investment scheduling and many other areas of operation research. In the continuous-time context, stochastic control theory has from the numerical point of view, been mainly concerned with Stochastic Differential Equations (SDEs in short). From the practical and theoretical point of view, the numerical developments for this class of processes are extensive and largely complete. It capitalizes on the connection between SDEs and second order partial differential equations (PDEs in short) and the fact that the properties of the latter equations are very well understood. It is, however, hard to deny that the development of computational methods for the control of PDMPs has received little attention. One of the main reasons is that the role played by the familiar PDEs in the diffusion models is here played by certain systems of integro-differential equations for which there is not (and cannot be) a unified theory such as for PDEs as emphasized by M.H.A. Davis in his book. To the best of our knowledge, there is only one attempt to tackle this difficult problem by O.L.V. Costa and M.H.A. Davis. The originality of this project consists in studying this unexplored area.